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This is an external summative evaluation report for the Self-Directed Workplace Distance 
Learning for Developmental Disabilities Workers Project, a partnership between The Center for 
Advanced Study of Education (CASE) of the City University of New York Graduate School and 
the Civil Service Employees Association, Inc. (CSEA), with the New York State Office of Mental 
Retardation and Developmental Disabilities (OMRDD) and the Government Office of Employee 
Relations (GOER) as helping organizations. The goals of the research component of the 
Distance-Learning Project were twofold. First, we wished to document changes in literacy skills 
and other relevant variables over the course of training. Second, we wished to link these changes 
to the training intervention. The purpose of this report is to present data and interpretations 
relevant to the achievement of these goals, and to draw conclusions where possible. An 
evaluation plan was created in the planning stages of the project. Many of the planned evaluation 
analyses were ultimately done, although as will be noted below, some aspects of the evaluation 
have been altered to accomodate changes in the project. 

The report will begin with a brief description of the research design and the measures used 
in the study. Results of the various analyses that examined changes in literacy skills and other 
measures will then be presented. The final section offers some conclusions based on these 
analyses. 
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Research Design 

The original research design called for a nonequivalent control groups design in which a 
distance learning group (the “treatment”group) would be compared to a control group of workers 
who had been wait-listed for participation in the distance learning program. Assignment to 
control versus learning groups would not be controlled by the researchers, and hence the groups 
would not be regarded as equivalent prior to the start of the program. Both pretest and postttest 
measures of literacy skills would be collected within each group. These measures are further 
described below, along with other measures used in the study. The outcome of the program was 
to be evaluated by comparing the change from pretest to posttest in the literacy skills measures 
between groups, using either direct comparisons of average change scores or the analysis of 
covariance with the pretest scores as covariates. This design was to be essentially replicated 
across multiple cycles of the program with different sets of workers. The final evaluation could 
aggregate data across cycles, or disaggregate if different results were expected as a function of 
cycle. 

The actual implementation of the study did not follow this plan after the first or second 
cycle of the project because of the difficulty in obtaining control group individuals. Nearly all of 
the control group individuals were in the first cycle of the program. Subsequent cycles either had 
no provisions for a control group, or did not have sufficient control group members with both 
pretest and posttest scores to permit group comparisons. By the end of the four cycles, the 
number of control group individuals with both pretest and posttest data was fewer than 10 on all 
of the major literacy outcome variables. Given the lack of a control group, we can evaluate 
changes in literacy skills for the participants following participation in the distance learning 
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program, but we cannot determine whether this change resulted from participation. 

A second difficulty in the actual implementation of the program was that large numbers of 
participants failed to have complete data on both the pretest and the posttest measures. Among 
the 253 participants for whom we have some data, fewer than 70 have both pretest and posttest 
data on any of the major literacy measures. The exact number of participants with both sets of 
data varies depending on the measure in question. The most frequent omission is the posttest 
data, although 102 participants are missing pretest data (90 of these are missing posttest data as 
well). The reasons for the missing data are unclear in most cases. The original plan in the event 
that an individual dropped out of the program was to collect information on the individual’s 
reasons for leaving. We were able to obtain dropout information on 40 participants who left the 
program. For the remaining participants who are missing data, the reasons for the missing data 
are unclear. Without both pretest and posttest scores on a given measure, it becomes impossible 
to evaluate changes in skills that might result from participation in the program. Fortunately, 
enough of the participants had complete data on the literacy skills measures to permit some 
evaluation of change. 

In addition to the main portion of the design, a secondary, smaller component of the 
design was included to look at longer-term changes in literacy skills that might result from 
participation in the distance-learning program. During the final three months of the study period, 
12 participants who received training during the first year of training were again interviewed 
regarding job performance and career advancement. These 12 individuals will be selected to have 
adequate variability on the pretest measures: four individuals were selected from the highest 
performing group on the pretest, four were selected from the average group on the pretest, and 
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four were selected from a low performance group. The data obtained from these participants will 
be denoted the “longitudinal data” in what follows. Although the total group size is too small for 
tests of statistical significance to have adequate power, some descriptive results for the group will 
be reported below. 

Measures 

Background Measures 

Background information was recorded for each individual that entered the project. This 
information includes the individual's age, gender, marital status, number of dependents, job title, 
job location, job tenure, whether full or part-time, the language used in childhood, the language 
used most often now, the highest grade completed in school, whether the individual has had any 
non-credit courses in reading or writing, or any other training that the individual has received. All 
of this information was gathered when the participant first entered the project, for both treatment 
and control individuals. 

Pretest and Posttest Measures. 

Pretest literacy measures were given to both treatment and control individuals during their 
initial entrance into the program. Posttest literacy measures were given at the end of the six 
month training period. The time required for either set of measures to be completed by a 
participant was about 1 1/2-2 hours. 

All literacy measures in this evaluation were customized to the job, and were locally 
developed. Each participant completed a test of reading comprehension at pretest and posttest, 
denoted the "Reading Test" in what follows. Examinees were given three reading selections, with 
each selection followed by a set of multiple-choice questions that refer to the selection. The 
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reading selections contain job-related material similar to that encountered on the job. A total of 
10 multiple-choice questions were given. Each examinee was given 20 minutes to complete the 
test. The examinee's score is calculated as an unweighted total across these 10 questions and then 
expressed as a percentage of the number of points possible. 

The next test was a direct writing assessment measure, denoted the "Writing Test" here. 

In this test, examinees were asked to write a description of their job as if they were describing the 
job to a coworker. Each examinee was given 20 minutes to write the essay. The essays were 
scored by two trained readers. The scoring system is an analytic system that rates the essay on 
five dimensions: content, organization, vocabulary, language use, and mechanics. Each dimension 
is scored on a four-point scale ranging from "very poor" to "excellent to very good". The total 
number of points given for each essay therefore ranges from 5 to 20. The two readers were 
instructed to rate an essay independently, and then to reach consensus if their ratings differ. If a 
consensus cannot be reached, a third reader read the essay. The final score for the essay is an 
average of the three reader's ratings, again expressed as a percentage of the total possible points. 

Each participant also completed a test of problem-solving skills, denoted the "Problem- 
Solving Test" here. In this test, the examinee was presented with a series of job-related scenarios. 
Each scenario consists of a problem situation typical of those that might be encountered on the 
job. The examinee must write a paragraph describing what he or she feels is the best way of 
resolving the problem. Four scenarios are presented in the test, and each requires a separate 
written response. The written responses were scored by readers who have been trained for this 
scoring task. "Best case" solutions to each scenario (and the scenarios themselves) were 
developed in collaboration with present and former staff members of the New York State Office 
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of Mental Retardation and Developmental Disabilities (OMRDD). The response to each scenario 
was scored on a four-point scale to indicate its similarity to the best case solution. Total scores 
across the four scenarios range from 4 to 16 points. The final score for an individual was 
expressed as a percentage of the total possible score. 

The final measure used in the pretest and posttest was a self-efficacy scale that concerns 
job-related competencies. This measure will be denoted the "Self-Efficacy Scale" here. In this 
scale, the examinee was presented with five work situations in which a general task is to be 
completed. Within each situation, the examinee was asked about the degree to which he or she is 
sure that various activities could be successfully completed. All of the activities concern different 
subtasks that must be performed in the situation. These subtasks involve the three different basic 
literacy skills of reading, writing, and math. The examinee responded to each question on a seven- 
point scale ranging from "Not at all sure" to "Very sure". A total of 18 questions were asked 
across the five situations. Total scores were calculated as sums of item scores across all five 
situations, and subtest scores were calculated to correspond to specific literacy skills (reading, 
writing, and math). All scores were expressed as percentages of the total number of possible 
points on a given scale. 

In addition to the measures to be completed by the participants, the supervisor of each 
participant was asked to complete some ratings of the participant at both pretest and posttest. 
These ratings were collected on the same time schedule as the other pretest and posttest 
measures. The ratings consist of 12 questions regarding how well the participant is able to 
complete various job tasks. The tasks involve basic reading, writing, and arithmetic skills. 

Ratings are given on a four-point scale, with an additional rating category for "not applicable" to 
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be used if the participant never does the task in question. These pretest and posttest ratings will 
be denoted the "Supervisor Ratings" in what follows. 

Along with the Supervisor Ratings, each participant was asked to rate his or her own task 
completion using the same set of 12 job tasks that were rated by their supervisor. The response 
scale is the same as that used by their supervisor. At posttest, the participant was also asked to 
rate the amount of improvement on the same four skills rated by their supervisor. These 
participant self-ratings will be denoted the "Participant Ratings" in what follows. 

Results 

Demographic characteristics 

The sample is mostly female (83%), with ages ranging from a low of 23 to a high of 68. 
The median age is 39. Almost 80% of the sample were bom in the United States, and nearly all 
speak English (96%) as the language spoken most often in the home. The racial status of the 
participants was assessed with two questions, one asking whether the participant is Hispanic, and 
another asking the participant to chose among the categories White, Black (African- American), 
Asian or Pacific Islander, American Indian or Alaskan Native, and Other. Some participants did 
not respond to either question, but based on the non-missing responses, the racial breakdown of 
the sample is as follows: 70 White, 118 Black, 8 Hispanic, 2 Hispanic/White, 5 Hispanic/Black, 1 
Indian/Hispanic, 1 Asian, 2 Indian, and 6 Other. 

The median number of years that the participants have worked at their current job is 8 
years, with the number of years ranging from less than one year to a maximum of 32 years. On 
average, the participants work 39 hours per week. The participants varied in their job titles, but 
four titles accounted for the majority of workers: Developmental Aide (84%), Food Service 
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Worker (8%), Resident Supervisor (3%), and Cleaner (3%). 

Among those participants who said that they had received education in the United States 
(N=199), 75% received 12 or more years of education. There were 30 participants who indicated 
that at least part of their education was obtained from foreign schools. Of these 30 participants, 
22 received 12 or more years of education from foreign schools. Participants were also asked 
about their training outside of regular high school coursework. There were more missing data for 
these questions than for the earlier questions about education, but the percentages of participants 
who endorsed each educational alternative, as percentages of the number of participants who 
responded, were as follows: College courses for credit (61%), Vocational courses (42%), 
GED/High school equivalency (3 1%), Adult basic education or literacy (12%), and Work-related 
training (86%). Clearly, there are a fair number of participants who have sought to supplement 
their education by alternative routes. 

The participants were asked to indicate the types of skills that are needed on the job from 
a list of such skills. The following are the percentages of participants who endorsed each skill. 
Read Instructions (98%), Receive spoken instructions in English (99%), Speak English (100%), 
Work as part of a team (99%), Write in English (99%), Use math (74%), Solve problems/use 
reasoning (98%). Apart from the use of math, it appears that nearly all of the participants feel 
that all of the skills are relevant to the job. The participants were also asked to rate their ability to 
perform various activities, using a four-point scale from “Poor” to “Excellent”. The following are 
the activities and the percentages of participants who rate their ability as either “Good” or 
“Excellent”: Read English (86%), Understand English (92%), Speak English (90%), Write in 




English (76%), Work as part of a team (94%), Use math (63%), and Solve problems/use 
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reasoning (77%). Although writing in English and solving problems are nearly universally 
regarded as necessary for the job, almost a quarter of the sample feel that their own ability in these 
areas is either “Poor” or only “Fair”. A good portion of the sample also appear to have doubts 
about their ability to use math, though this activity is less necessary for the job. 

Participants were asked about what they would like to get out of the Distance Learning 
course, from among a list of possible goals. All of these goals are endorsed by over half the 
participants: Communicate better on your job (89%), Improve communication for everyday life 
(91%), Help a child or children with homework (76%), Go on to further studies (87%), and 
Develop your career (92%). The participants were also asked to indicate any possible barriers to 
their success in the Distance Learning program. About 63% of the participants indicate that they 
perceived no barriers to success. Another 29% of the participants feel that “time” is a potential 
barrier, as in “released time” or “time to practice”. A variety of other barriers are listed by other 
participants, such as “child care”, but the numbers of participants here is small. 

Results for outcome measures 

All participants were given a battery of tests in reading, writing, and problem-solving 
before beginning the program, and at the end of their participation in the program, as mentioned 
earlier. The two administrations will be denoted the “pretest” and “posttest”, respectively, in 
what follows. The score of interest in each case is the percentage of points achieved by the 
participant out of the total number of possible points for the test. Our interest lies in the change in 
the score from pretest to postest. In addition to these test scores, participant self-ratings and 
supervisor ratings were collected at both pretest and posttest, each expressed as a percentage. 
Finally, participant self-efficacy measures were taken at both pretest and posttest for reading, 
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writing, and math. These are also expressed as percentages. Changes in the ratings and self- 
efficacy measures were examined for those participants having data at both pretest and posttest. 
A substantial number of participants were missing data either at pretest or at posttest on all 
measures and test scores however, as mentioned above. Any analyses of change scores were 
based on those participants who had complete data for the test being examined. 

Reliability . A number of the measures of interest had multiple items, permitting an 
evaluation of internal consistency reliability (alpha coefficients) at the pretest and posttest 
measurements. The following are the results of these analyses. The results are based on the 
maximum number of participants who had data on each measure at a given time period. This 
involved from 146 to 151 participants at pretest, depending on the measure, and 76-77 
participants at posttest: 



Test 


Alpha at pretest 


Alpha at posttest 


Writing 


.88 


.87 


Problem-solving 


.89 


.80 


Self-Eff Reading 


.88 


.93 


Self-Eff Writing 


.88 


.91 


Self-Eff Math 


.68 


.65 


Total Self-Eff 


.93 


.95 


The Self-Efficacy 


in Math scales had substantially lower alpha values 



Math scales had fewer items (three) than any of the other scales. The alpha values are generally 
acceptable. 
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Pre-Post changes. The following are the sample sizes, means, standard deviations, and t- 
values for the post-pre change scores on the three subject tests: 



Test 


N 


Mean 


m 


t-value 


Problem-solving 


64 


6.06 


25.73 


1.88 


Reading 


63 


-2.90 


19.85 


-1.16 


Writing 


61 


1.15 


13.88 


.64 



Note that the scores are expressed as percentages, so that the change scores are changes in 
percentages. Scores on problem-solving and writing showed an increase on average between 
pretest and posttest. The change in writing scores did not approach statistical significance, but 
the change in problem-solving scores was nearly significant at the .05 level. The reading scores 
did not show an increase on average from pretest to posttest. 

The above statistics were recalculated for participants in cycles 2, 3, and 4, omitting the 
cycle 1 participants. The purpose here was to see if better results were forthcoming in the later 
cycles when some of the inevitable difficulties facing the start of a project of this scope would 
have been eliminated. The following results were obtained for the three literacy measures: 



Test 


N 


Mean 


SD 


t-value 


Problem-solving 


34 


7.35 


30.08 


1.42 


Reading 


33 


-5.30 


23.38 


-1.30 


Writing 


32 


4.66 


15.13 


1.74 




These results parallel those obtained with the full sample that included cycle 1 . The average 
changes are larger, but are still negative for the Reading test. Note that the t-values are not larger 
however, partly due to the smaller sample in these analyses. 
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On the self-efficacy measures, the following are the sample sizes, means, standard 
deviations, and t-values for the change scores on each of the subject-specific measures and a total 



self-efficacy 


measure: 








Measure 


N 


Mean 


£D 


t-value 


Reading 


65 


6.15 


14.32 


3.46 


Writing 


65 


6.78 


13.70 


3.99 


Math 


65 


5.83 


20.11 


2.34 


Total SE 


65 


5.60 


12.92 


3.50 



The total self-efficacy measure is a composite of the reading, writing, and math self-efficacy 
scores. Each of the self-efficacy measures shows a statistically significant increase from pretest to 
posttest. These results indicate that the participants’ feelings of self-efficacy increased in each of 
the subject areas after participation in the distance learning program. 

Similar results were found for the changes in the participant self-ratings and the supervisor 
ratings. Here are the descriptive statistics: 



Measure 


N 


Mean 


SD 


t-value 


Participant 


57 


4.95 


14.28 


2.62 


Supervisor 


42 


8.29 


17.04 


3.15 



The participants showed statistically signficant gains on both of these rating measures. The gains 
on the supervisor ratings were especially useful, as these are based on raters who are not either 
teachers or learners. 

For reference purposes, the following table gives the pretest and posttest means on all of 
the above outcome measures for participants with complete data on each measure (for all cycles 
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combined— the sample sizes are given above), along with the correlation between the pretest and 



posttest scores: 








Measure 


Pretest 


Posttest 


Correlation 


Reading 


87.35 


84.44 


.297 


Writing 


70.90 


72.05 


.625 


Prob-solving 


63.02 


69.08 


.211 


Partic rating 


76.37 


81.32 


.330 


Superv rating 


75.88 


84.17 


.483 


Reading Self eff 


72.74 


78.89 


.563 


Writing Self eff 


70.23 


77.02 


.550 


Math Self-eff 


73.31 


79.14 


.358 


Total Self-eff 


72.37 


77.97 


.542 



In addition to the test scores, the participant’s instructor provided information on the 
number of contact hours, the number of email contacts, the number of regular mail contacts, and 
the number of units completed for each participant. These four variables provide information 
about the amount of work completed by the participant, as well as the extent of contact between 
each participant and the instructor. The variables will be denoted the “contact variables” in what 
follows. Scores on the contact variables were available for nearly all participants. Correlations 
between scores on the contact variables and the change scores for the above three tests were 
calculated in order to evaluate whether the contact variables might moderate the test performance 
change. No statistically signficant correlations were found between the two sets of variables. The 




correlations were: 





Problem-solving 


Reading 


Writing 


Contact hours 


.155 


-.052 


-.131 


Email 


-.026 


-.171 


-.021 


Hmail 


.062 


-.026 


.023 


Units 


.035 


-.059 


-.049 



There does not appear to be any systematic relationship between the contact variables and the 
changes in test performance. It is possible that more contact time was spent with students who 
were having difficulties, and that the greater time spent with these students did not directly 
translate into greater gains in test performance. 

In addition to the contact variables, the instructor’s rated each participant on the degree of 
“improvement” shown by the participant in six areas, along with an overall rating. Ratings were 
on a four-point scale, ranging from l=No improvement to 4=A lot of improvement. The sample 
sizes, means, and standard deviations for the ratings were as follows: 



Rating name 


N 


Mean 


SD 


Understand technical vocab 


146 


2.34 


1.09 


Reading comprehension 


146 


2.35 


1.13 


Spelling 


148 


2.20 


1.06 


Relaying accurate info in writing 


146 


2.36 


1.09 


Using basic math 


113 


2.07 


1.05 


Oral communication skills 


153 


2.41 


1.17 




Overall improvement 



151 



2.36 



1.10 
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For all ratings, the average rating fell between 2 (“some improvement”) and 3 (“good 
improvement”). The ratings were correlated with the change scores on the three tests. No 
statistically significant correlations were found, even between corresponding variables (e.g., 
Reading score and rating on reading comprehension). The largest correlation found was between 
the rating on “using basic math” and the change in problem-solving score: the correlation was 
.189. It appears that the instructor’s impressions of the improvements in the participant’s skills 
did not agree with the change in test scores. It should be noted that the ratings themselves did not 
reveal much discrimination among the seven different rating variables. The intercorrelations 
among the seven ratings ranged from a low of .73 to a high of .91. All correlations were 
statistically significant. The most likely explanation for this pattern is that the instructor’s ratings 
were influenced by a general halo effect. 

Results for longitudinal data. 

The longitudinal data involved 12 study participants, divided into three groups (low, 
medium, high) based on their pretest performance. Four participants were interviewed in each of 
the three groups. The results of the analyses of these data are reported here. 

The 12 participants are all females, with the ethnic distribution being 6 Black, 3 White, 1 
Hispanic, 1 Indian, and 1 Other. The participants span five job categories: 7 Developmental 
Aides, 2 Food Service Workers, 1 Food Service supervisor, 1 Cleaner, and 1 Keyboard Specialist. 
Eleven of the 12 participants indicated that they are in the same job that they began with. The one 
individual who is in a different job did not respond when asked about her current job title. Five of 
the participants indicated that their responsibilities have changed, but only two of these persons 
described how the responsibilities had changed. Both indicated that they “work more 
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independently.” 

Three of the 1 2 participants indicated that they are enrolled in school, two in college 
courses and one in computer classes. Eight of the participants said that their career goals have 
changed. When the 12 participants were asked about their goals before the Distance Learning 
Program, only five responses were given. Three participants said that they were “not sure; just 
working”, one participant “wanted to be a DA”, and one person “didn’t think she could go to 
school.” It appears that these 12 participants did not have clear goals prior to Program. When 
asked about current career goals, the responses were more frequent and varied. The most 
popular reponses were “retire” (2 persons) and “college” (2 persons). All of the other 1 1 
categories of response received one response, except option 9 (GED), which no respondent 
included as a goal. 

When asked about the effect of the Distance Learning Program on the ability to do the 
job, the most popular response was “writing, writing better, more accurate” (6 persons). Other 
popular responses were: communicate better with other workers (3 persons), clarified roles and 
responsibilites of DA (3 persons), communicate better with parents (2 persons), and communicate 
better with clients (2 persons). Finally, when asked for other comments about the program, seven 
participants said that it was a “good program; enjoyed program” and six persons said that they 
had “good communication/rapport with instructors/advisors.” Three persons expressed some 
problem with the program. One person would like to take Part 2 of the program, and one person 
indicated that she is “more confident.” 

The change scores for the 1 2 participants on the tests given at pre and posttest were 
examined. The sample is too small for formal significance tests, but we can note whether the 
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majority of the individuals showed gains in scores. We do not have complete data on all 12 
participants. The numbers of participants with both pre and post scores ranged from a low of six 
(participant rating) to a high of eight (writing, total self-efficacy, problem-solving). We have 
nearly complete data for the “high” participant group, and fairly complete data for the “low” 
group. Data are scarce on the “medium” group. For those with complete data, the average 
change was positive on three of the outcome measures: supervisor rating, writing, and problem- 
solving. All other measures, including the self-efficacy measures, showed negative changes from 
pre to post. 

Conclusion 

The results of the data analyses do not lead to unequivocal conclusions about the effects 
of participation in the Distance Learning Program, but they are suggestive. As indicated by the 
three literacy measures of Reading, Writing, and Problem-solving, there was evidence of some 
gain in performance on Writing and Problem-Solving, although the gain did not reach statistical 
significance. Surprisingly, there was no evidence of positive gains in reading performance. In 
fact, scores on this measure decreased from pretest to posttest on average. It is unclear whether 
this result indicates some difficulty with the measure itself, or whether the instruction in reading 
was simply unsucessful in creating better performance. The Reading measure’s reliability is 
unknown, as it is not a multi-item measure and no reliability is available for the scoring of the 
measure. The participants’ feelings of self-efficacy in reading did increase following participation 
in the program. One possible explanation for the Reading result is that the pretest score levels on 
Reading were higher than on the other literacy measures, and so there may have been a ceiling 
effect that prevented dramatic gains in performance. 
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In an effort to see whether measures of program activity were associated with changes in 
literacy skills, we correlated measures of number of contact hours, amount of email and regular 
mail, and number of units completed with the change scores on the three literacy measures. No 
meaningful correlations were found. This result seems to indicate that the explanations for the 
results on the changes in literacy skills are not to be found in the participant’s activity levels. 

We solicited ratings from the instructors on the improvement shown among participants 
following participation in the program. These ratings indicated that, on average, at least some 
improvement was shown in all of the skills that were rated. The ratings do not show meaningful 
correlations with the gains in the three literacy test scores however. It is unclear whether the 
ratings themselves were sufficiently discriminating among the various performance dimensions. 
The minimum correlation between any pair of rated dimensions was .73, and many correlations 
ranged above .90. We can conclude from these ratings that the instructors believed that the 
average participant benefited from participation in the program. Finer conclusions regarding 
differences in improvement between specific dimensions are probably not warranted, given the 
general halo found in the ratings. 

Clear evidence of gains in participants’ feelings of competence and self-efficacy were 
found, as statistically significant gains in all three of the self-efficacy measures were present. It 
seems that regardless of the actual increase in skill levels, the participants felt more capable 
following participation in the program. These results are consistent with the participants’ self- 
ratings and the ratings of the participants’ supervisors, both of which showed significant gains 
from pretest to posttest. It appears that the supervisors felt that the employees did benefit from 
participation in the distance learning program. 
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The longitudinal data provide a more qualitative assessment of the program, as the number 
of participants involved in this phase of the design was small (12 participants). These individuals 
reported that their writing skills improved as a result of participation in the program. This report 
is consistent with the literacy test data for these participants, as the average gain was positive on 
the writing measure. More generally, these participants felt that their communication skills 
improved also. The majority of the 12 participants felt that the Distance Learning Program was a 
positive experience. 

The original research design, had it been executed, would have permitted firmer 
conclusions about the impact of the program by including a control group for comparison 
purposes. In spite of the omission of the control group, there is some evidence that the 
participants in the Distance Learning Program benefitted from their participation, either in terms 
of increased skills, or in terms of increased confidence and feelings of self-efficacy. 
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